Automatic Generation of Multimodal Weather Reports from Datasets

Weather reports are created in various modes natural language text, specialized language text, tables and maps. The system presented allows the user to define his needs of weather information and requirements on the form of presentation. The system analyzes a dataset obtained through specific procedures of forecasting or observation, plans the product according to the user requirements and generates its components. Special emphasis is placed on the coherence of the report by investigating the rhetorical structures observed in this kind of text and the coordination between a map and a text specifying it. The method of generation is a knowledge-based one with three types of knowledge employed in the system terminological, rhetorical and grammatical . A prototype has been implemented and tested with original datasets. 1 I n t r o d u c t i o n The generation of information products stepped into a new phase characterized by the intensive application of artificial intelligence, computat ional linguistics and other modern information technologies. Currently, various data are collected into databases and specific procedures are applied for processing those data into forecasts, analyses, surveys and other types of information products. Usually, those products are in numerical form which is unsuitable for the general audience and even for many specialists. Therefore this data has to be converted into a human-oriented mode such as natural language (NL) text, tables, maps, diagrams. The automatic conversion requires formalizing the process a problem which nowadays cannot be attacked successfully except by gathering, coupling and employing various types of knowledge common sense, about the subject domain, grammatical , etc. In this paper, we report on a study of the automat ic generation of mult imodal weather reports from observed or predicted data. This particular problem is significant both from a practical point of view (various weather reports are to be made every day in many weather centers all over the world) and for its scientific aspects (it manifests the basic features of the generation of verbal *This work has been partially supported by the Ministry of Education and Science and the Bulgarian Academy of Sciences. reports from data). Our work relates closely to three areas: NL generation, mult imodal documents and weather information processing. The communicative act performed by the system is the description of an observed or predicted situation. Other works that consider analogous communicative acts are (Davey, 1979) on the description of tic-tac-toe games, (Kukieh, 1983) on the generation of market reports, (Andr~ el al., 1988) about simultaneous commenting on a soccer game recorded as a sequence of digitized video frames. In our case the situation to be presented is coded into a dataset obtained through routine procedures of weather forecasting or observing. Our approach to NL generation follows the basic steps as described by McDonald (1987), viz. selection of the content portions that are to be communicated to the user, planning the text by adoption of the most suitable rhetorical schemas, realizing the discourse plan as a surface structure and its rendering as a text. The goal and the context of the ut terance are specified by the user together with parameters concerning the precision of the information and the message length. We place special emphasis on the content production component which scans the dataset and extracts assertions from it, as well as on the rhetorical structures observed in weather reports. Recently an increasing interest has been observed in the processing of mult imodal documents, the research being focused on the coordination between the different modalities (NL, graphics, video images, pointing). Some projects with intensive research in this area are XTRA (Algayer et al., 1989), C O M E T (Feiner and McKeown, 1990), ALFresco (Stock, 1991). To a large extent this aspect of our project was inspired by the WIP project (Wahlster el al., 1991) in which the coherence of mUltimodal discourse is investigated and common sense knowledge is employed in the coordination between the textual and the graphical components of instructions for the use of domestic appliances. We consider the case of supplementing a weather map with a verbal note specifying those content portions that cannot be presented on the map or whose graphical presentations distort the original information. The system discovers such deficiencies of the graphical presentation and generates a verbal comment on the map. There are various projects concerning the production of weather reports, each of them setting specific goals